Visual codes such as UPC symbols are commonly used to associate a unique identifier (a number or binary bit code) with a given physical object. This unique identifier is detected by employing image processing techniques that analyze the arrangement of parts of the visual pattern printed on the object. In designing such visual coding schemes, it is important to consider whether the pattern may be easily detected by imaging the object, the ease with which the identifier may be recovered from the pattern once the pattern is located in the image, and various features of the visual code such as the number of bits encoded, error detection and/or correction, rotation dependence or invariance, etc.
An application in which a visual code must be decoded can often impose constraints on the form of the visual code used to provide an identifier for an object. For example, the object on which the visual code is applied may be relatively small, so that the visual code has to be compact. Furthermore, the pattern employed for visual coding must be perceptible to the imaging system used to read it within the resolution constraints of that system. If the small object has a particular shape, for example, a round “foot print” on the surface where the visual code is to be detected, it would be desirable to use a circular visual code that generally matches that round shape of the object. Similarly, if a small object is rectangular or oval, it would be desirable for the shape of the object to assist in determining the orientation of the visual code, to facilitate decoding the pattern used in the code, to identify the object.
Visual codes used for UPC bar codes on products sold in markets or other retail establishments are typically read using a coherent light laser source to scan the bar codes. Such laser scanning systems have a relatively high resolution and employ laser sources with sufficiently high intensity to scan the bar codes without interference from ambient light. In contrast, it is much more difficult to decode a visual code on an object within a image, particularly, when the image is of a relatively large surface, such as the display surface of an interactive display. The imaging system must first detect an object in the image of the surface and then decode the pattern of the visual code applied to the object.
Interactive displays that employ visual codes to identify objects are known in the art. For example, an interactive display platform was developed in the MIT Media Lab, as reported by Brygg Ullmer and Hiroshi Ishii in “The metaDESK: Models and Prototypes for Tangible User Interfaces,” Proceedings of UIST 10/1997:14–17. The metaDESK includes a near-horizontal graphical surface used to display two-dimensional geographical information. A computer vision system inside the desk unit (i.e., below the graphical surface) includes infrared (IR) lamps, an IR camera, a video camera, a video projector, and mirrors. The mirrors reflect the graphical image projected by the projector onto the underside of the graphical display surface. The IR camera can detect reflected IR light from “hot mirrors,” comprising a material backing on the undersurface of objects called “phicons” that are placed on the graphical surface. In response to the IR camera detecting an IR reflection from this material, which is transparent to visible light, applied to the bottom of a “Great Dome phicon,” a map of the MIT campus is displayed on the graphical surface, with the actual location of the Great Dome in the map positioned where the Great Dome phicon is located. The paper does not provide any teaching or suggestion of using an IR pattern and does not teach or explain details concerning the manner in which it is detected and identified, other than noting that for each vision frame, the software produces “a list of unidentified ‘blobs’ extracted from the scene.”
A similar approach to sensing objects on a display surface is disclosed in several papers published by Jun Rekimoto of Sony Computer Science Laboratory, Inc. in collaboration with others. These papers briefly describe a “HoloWall” and a “HoloTable,” both of which use IR light to detect objects that are proximate to or in contact with a display surface on which a rear-projected image is visible. The rear-projection panel, which is vertical in the HoloWall and horizontal in the HoloTable, is semi-opaque and diffusive, so that objects become more clearly visible as they approach and then contact the panel. Although mention is made of using IR detectable patterns on objects for identifying them, again, no details are provided about the IR pattern or about how the IR pattern is detected to identify the objects.
Many types of visual codes are known in the prior art. However, given the limitations on resolution and other constraints mentioned above, it is likely that many of the prior art visual codes designed to work with high resolution scanners will not be suitable for use with a vision system that must use reflected IR light in an image of an entire display surface to read a visual code applied to a relatively small object in the image. Accordingly, it would be desirable to employ visual codes that are appropriate for the shape and size of the objects to be identified, and which can be identified with a vision sensing system having relatively limited resolution. A suitable encoded pattern should be sufficiently easy to decode to enable realtime decoding while an object having the encoded pattern is being moved about on a surface. It would be desirable that the pattern provide a relatively high contrast IR in the image of the surface, to enable the pattern to be decoded through a diffusing surface, since many prior art encoded patterns require too great a resolution to be usable on such a surface. A further desirable characteristic of a suitable encoding pattern is that it have an innate orientation, to reduce the search space required to find the encoded pattern on a surface. For similar reasons, the encoded pattern should preferably be recognizable as a single connected component, to facilitate detecting and decoding it. The prior art does not address many of these problems, which are more specific to a diffusing display surface and vision sensing system, as described above.